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(57) Abstract 

A method and apparatus for a computer and telecommunication network which can receive, send and manage information from or to 
a subscriber of the network, based on the subscriber's configuration. The network is made up of at least one cluster containing voice servers 
which allow for telephony, speech recognition, text-to-speech and conferencing functions, and is accessible by the subscriber through 
standard telephone connections or through Internet connections. The network also utilizes a database and file server allowing the subscriber 
to maintain and manage certain contact lists and administrative information. A web server is also connected to the cluster thereby allowing 
access to all functions through Internet connections. 
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INTERNET AND TELEPHONY BASED MESSAGING SYSTEM 
FIELD OF THE INVENTION 

The present invention relates to a telecommunication system, and more 
5 specifically, the present invention relates to a network system based on internet, 

computer and telecommunication standards, utilizing internet and computer technology, 
a graphical user interface, integrated telecommunication applications and interactive 
voice recognition technology, facilitating the computation and telecommunication of 
voice and electronic data. 

1 0 BACKGROUND OF THE INVENTION 

Individuals and business people seek to communicate with each other, obtain 
useful information, interact commercially and entertain themselves in an increasingly 
mobile society. In order to fulfill these needs, one requires the ability to send and 
receive messages, access information and entertainment, conduct business transactions, 

1 5 organize daily schedules and stay in touch with homes and offices from almost 
anywhere, at any time, as easily as making a telephone call. 

Continued demand for products and services that address these needs is 
evidenced by the increasing number of electronic devices, and the explosive growth of 
the internet and network services. Advances in wireless telecommunication 

20 technologies led to the development of such devices as personal digital assistants, and 
enabled the growth of paging and cellular telephone networks. Devices such as 
notebook and sub-notebook computers with modems (both wireline and wireless) have 
allowed mobile professionals to connect to their PCs from almost any location, as well 
as to access on-line information and electronic mail services while traveling worldwide. 

25 In addition, communication and information needs have stimulated the growth 

of the internet, on-line networks and corporate intranets. These networks now host a 
variety of services such as e-mail, database searching, conferencing, electronic 
commerce, games, software libraries and electronic newspapers and magazines. 
However, despite the proliferation of communication devices and the development of 

30 the internet, on-line networks and corporate intranets, significant barriers remain to 

fulfilling user needs for access to and management of personal, professional and public 
information. 
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The hardware designs and software technologies which enable today's 
communication are complex. Information, though widely available, can be either 
inaccessible or accessible only by navigating through a host of phone systems, 
operating system platforms, databases and networks. As a result significant amounts of 
5 time and effort are required of those who use and depend on these devices, networks 
and services to communicate and obtain information. 

The problem of accessing and processing all of the information available from 
communication devices, networks and services is particularly acute for mobile business 
professionals. Today's mobile professional, working out of the home or small office, 
10 may have a cellular phone, a pager, a computer, a fax machine, an electronic mailbox 
on the internet, and a voice-mail service. Whether on the road, in a plane or at the 
office, success for the mobile professional depends in large part on the ability to easily 
and quickly access, sort through and respond to the messages delivered to each of these 
communication devices, and to obtain information necessary to the conduct of business 
15 from proliferating networks and services. 

SUMMARY OF THE INVENTION 

The present invention is a network system, which is based on internet, 
computing and telecommunications standards, utilizing computer and internet 
technology, an innovative graphical user interface, integrated communication 

20 applications and interactive voice recognition technology. The present invention is a 
unified messaging service which will be accessible from any standard communication 
device (telephone, computer or internet), and will give the user intuitive voice 
command of personal, professional and public information. 

This unified messaging service is a useful tool to those whose time and 

25 resources are limited and for whom communication is critical, such as mobile business 
professionals in the small office, home office market. The mobile business professional 
must maintain access to personal and professional information and developments, 
respond to customers and communicate with colleagues, family and friends at any time 
and from any location. The unified messaging service is designed to meet these 

30 objectives by offering a single point of access to all communications, integrated with 
personal information management tools and customized public content delivery. 
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Small office, home office professionals, most of whom do not have access to 
dedicated information management systems or the benefit of administrative support 
staff, may derive unique value from this shared network solution. 

BRIEF DESCRIPTION OF THE DRAWINGS 
5 Preferred embodiments of the invention are explained below with references to 

the accompanying drawings in which: 

Figure 1 is a functional block diagram of the hardware platform embodying the 
present invention; 

Figure 2 is a functional block diagram of the internet platform embodying the 
1 0 present invention; 

Figure 3 is a general application flow chart embodying the present invention; 

and 

Figure 4A-4R are detailed application flow charts embodying the present 
invention. 

1 5 Figure 5 A-5B are the functional block diagrams for the computer telephony 

platform and network architecture embodying the present invention. 

Figure 6 is the software architecture embodying the present invention. 
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
The present invention is a network system that works, among other things, as 
20 your voice-mail system, using an 800 888 number that receives all your calls, faxes, 
pages and e-mail. The system provides a contact database facilitating the placing of 
calls, screening of calls, and tracking you down wherever you are. 

Regardless of how the message was transmitted, every message is delivered to 
you through a single source, whether telephone, internet or computer. You can also 
25 review your latest call records and billing information and change or add to your phone 
listings, contact numbers and service preferences through this source. 

The present invention utilizes a fault resilient redundant system, residing in two 
separate sites. There are multiple T-3 (45 Mbps) facilities going into each of those sites. 
Both sites sit on a fiber-channel fiber optic loop, which is theoretically "self healing" in 
30 case of a disaster and can route a phone call to either facility, depending on where the 
route is broken. 
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The configuration at each site is a cluster of servers. Figure 1 is a block diagram 
of the hardware platform showing one of these clusters 10 which consists of some voice 
servers 12, some mirrored Sybase database servers 14, and some web servers 16 where 
the web access and services are located. The computers (not shown) are 200 MHz Intel- 
5 based 191 1 rackmount servers running a combination of Solaris and SCO UNIX 
operating systems. The voice server 12 includes certain functions, such as telephony, 
automatic speech recognition, text-to-speech, conferencing, etc. Subscribers are 
connected to these clusters by either normal telephone connections or by internet 
connections. 

10 Each cluster can serve about 10,000 customers. The system is open-ended 

allowing for the addition of subscribers as needed. Further the design facilitates the 
easy addition of hardware and software. 

Figure 2 shows a block diagram of the internet platform. The system may be 
accessed via the internet instead of a normal telephone connection (voice or touchtone). 

15 The subscriber can access the web server 1 8 through an internet firewall 20. The 

subscriber merely enters the system's web site and then can access his account through 
a security program. Once in his account, the subscriber can access various features such 
as playing voice mail, reading e-mail and faxes, managing contacts and schedules, 
among other services. Further, the system can provide a private line 22 for corporate 

20 and other subscribers which can enter the web server 1 8 through an extranet firewall 24. 

Figure 3 shows a general application flow chart for an incoming call 26 to the 
system. The incoming call 26 whether by telephone connection or internet connection is 
directed either to customer service 28 (through an automatic call distribution-ACD), fax 
30, call routing 32, or conference 34 function. Call routing 32 is determined by the 

25 incoming call control 38, previously set by the subscriber, and can include a message 
and notification to the subscriber, call screening, call waiting, and the transfer of the 
call. Further, if the incoming call 26 is a subscriber 36, the system will provide 
additional functions to the subscriber, such as controlling incoming calls 38, including 
call screening, call waiting and transferring calls. The subscriber may set priorities to 

30 certain callers allowing only certain callers to reach the subscriber while all others are 
sent to voice-mail to record a message for playback later. 
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Also, the subscriber will be able to manage 40 his account. Managing 40 
includes setting the options for telephone calls 42, setting the options for sending and 
receiving messages, faxes and e-mail 44, managing the database containing the 
subscriber's contacts and other lists 46, and setting the parameters for outgoing call and 
5 setting up conferences 48. 

Figures 4A through 4R show a more detailed flowchart of the present invention. 
Figure 4 A shows the First Menu encountered by a subscriber or an individual calling 
the system's 800 telephone number. The caller is explained the different options and 
then the system, if requested, attempts to recognize the subscriber or party's name or 
1 0 extension that the caller is trying to reach. 

Figure 4B shows the Party Menu which allows the caller to begin the transfer to 
the party hr is trying to reach, leave a message, or, if the caller is a subscriber, open his 
mailbox. Figure 4C shows the Mailbox Menu which allows the subscriber to listen to 
his messages, leave a message, make a call (using the database), dial a telephone 
1 5 number, request the system to follow the subscriber, transfer calls, and set other 
personal options, including the greeting, security codes, etc. 

Figure 4D shows the Listen to Messages Menu which allows the subscriber to 
listen to, forward, or return the message, along with other options such as adding a 
contact, etc. Figure 4E shows the Message Menu such as forwarding the message, 
20 providing caller identification information, and updating the contact database. 

Figure 4F shows the Return a Call Menu which will attempt to use caller 
identification to return the call. 

Figure 4G shows the Leave a Message Menu which allows the caller to leave a 
message to the intended party. Figure 4H shows the After Record Message Menu which 
25 allows the caller to listen to the message, re-record the message, send the message, or 
cancel the message. Figure 41 shows the Make Call Menu which allows the subscriber 
to dial a number, cancel, or attempt to obtain a contact and transfer to that contact. 

Figure 4J shows the Dial a Number Menu which allows the subscriber to dial a 
telephone number. Figure 4K is the Call Transfers Menu which allows the subscriber to 
30 transfer incoming calls to either his caller identification, a telephone number, or some 
other number of choice. Figure 4L is the Follow Me Menu which allows the subscriber 
to receive calls wherever he may be located. Figure 4M shows the Notification Menu 
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which allows the subscriber to be notified either by pager or e-mail in certain situations. 
Figure 4N shows the Telephone Numbers Menu which allows the subscriber to dial a 
telephone number either by depressing the telephone keypad or speaking the number. 
Figure 40 shows the Record a New Greeting Menu which allows the subscriber 
5 to record, listen, or re-record a greeting which a caller will hear upon reaching the 

subscriber. Figure 4P shows the Name Recording Menu which allows the subscriber to 
set a particular name in the system. Figure 4Q shows the Personal Options Menu which 
allows the subscriber to change the security code, the name recording, and the personal 
greeting. Figure 4R shows the Set Security Code Menu which allows the subscriber to 

10 set a new security code. 

Figure 5 A shows the computer telephony platform in accordance with Figure 1 
in more detail. Figure 5B shows the network architecture in accordance with Figure 2 in 
more detail. Figure 6 shows the software architecture utilized in the present invention. 
The system provides three ways for the subscriber to handle his 

15 communications. First and foremost is the voice recognition software using natural 
voice recognition (phonemes based), not pattern based as many of the current systems 
utilize. Therefore, the system does not have to be trained to identify your voice. Second, 
the subscriber may use the standard telephone touchtones. And third, the subscriber can 
utilize the internet to access a secure web site. 

20 The system can act as a "follow me" service, capable of tracking you down 

whether you're at home, the office, airport, in a hotel room, in another office or in your 
car. The subscriber can configure the system to follow him, or the subscriber can 
disengage the follow me with a simple command. It also lets you know who is calling 
before you accept the call. 

25 As for Web connections, you can boot up your computer, sign onto the internet, 

go through the system's secure web site and play your voice mails, read your email and 
faxes, manage your contacts and schedule conferences. The website thus becomes a 
unified messaging system. If you get a message from somebody who's not in your 
address book and you want to call him or her, you just go to the web site again and add 

30 a contact. 
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If you're on the secure website and you walk away, the system even times out 
after a few minutes so, no one can sit at your computer and obtain your messages and 
contacts without your permission. 

While on the system's web site, you'll be communicating over the internet via 
5 the Secure Socket Layer (SSL), a transport level technology (developed by Netscape) 
for authentication and data encryption between a Web server and a Web browser. SSL 
sends data over a "socket," a secure channel at the connection layer existing in most 
TCP/IP applications. 

The system also utilizes text-to-speech so you can listen to your e-mail over the 

10 phone, or the subscriber can display his e-mail on his computer where he can reply to it 
and send e-mail to his contacts with audio (using audio attachments). Further, using 
Vosaic's internet audio program, the system can deliver voice mail via streaming audio 
over the internet using Java without a plug-in. 

During a session on the web site, using text-to-speech, you could type in a name 

1 5 like "Harry Newton" and his phone number so that, when you eventually use the 
system the phone and you say, "call Harry Newton at the office," the system will 
recognize the name and dial the number you keyed in previously. Based on what you 
input now ("Harry Newton") is what the system utters later and compares the phonemes 
of what you input now to what you'll be speaking later. 

20 The system also facilitates conference calls by allowing the subscriber two ways 

in which to set up a conference call. First, inbound conference calls can be set up for 32 
people if you've registered a pre-scheduled meeting time. Subscribers call in to the 
system to get an assigned conference number. The conference participants then call in 
and asks to "join the conference". The system then asks for a conference number, which 

25 is entered via touchtones. All the parties are then connected and the conference begins. 

Second, outbound conference calls can also be set up, even if you're "on the 
fly." Whether you're at your desk, at a hotel or even a phone booth, you can simply dial 
up and tell the system to "set up a conference call." Then you only have to dial up all 
the various parties to connect each of them to your conference. 

30 The present invention is a compilation of hardware and software, including 

voice processing using Dialogic's SCSA extended bus and board-level resources, 
Purespeech speech recognition running on Dialogic Antares boards, and database 
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management using Sybase System 10 and 1 1 . Further, the system utilizes data 
networking, particularly TCP/IP and distributed systems, object oriented design and 
programming, multi-processing with Intel hardware, SCO UNIX and Solaris operation 
systems, Java and JavaScript languages, US and international long distance protocols, 
internet and web protocols, credit and payment processing, a help desk, customer 
service system, and network and service management. 

For the voice and fax processing servers, the system relies on Dialogic 
hardware, including, voice boards, Antares cards (for speech recognition and fax) and 
digital switching cards. The system also uses the SCSA extended bus. The Purespeech 
ASR algorithms, which are used for speech recognition, run on the Antares cards 
plugged into the computers running the Solaris OS. 

The system combines state-of-the-art speech recognition, computer and 
telephony technology. Along with the ability to recognize an extensive set of simple, 
intuitive, speaker-independent speech commands and respond by performing a wide 
variety of complex tasks. Each subscriber has their own secure Web page on which all 
the features can be accessed, allowing almost every aspect of the present invention to be 
maintained on-line. 

Subscribers can use voice commands to (1) build, edit and manage their contact 
lists, (ii) review, play back, read, reply to and/or reroute voice mail and e-mail, (iii) 
schedule conference calls with 800 number access, and (iv) maintain an itemized 
calling log listing all calls with a running total of all charges. 

The system also transfers calls as instructed or upon command will follow the 
subscriber according to predetermined contact numbers for office, home, cellular, pager 
or other designated locations. Every time a subscriber calls in, the system logs the 
originating number for the inbound call and then uses that number to re-contact the 
subscriber. 

In accordance with the present invention, every subscriber or participant is 
given their own 800 number that is then used by all contacts calling in to the subscriber. 
The 800 number can also be used in scheduling conference calls for up to 32 
participants on a given day and time. Each participant uses the 800 number and calls in 
to join the conference call, thereby effecting call conferencing for a fraction of the cost 
otherwise incurred. 
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Basically, the system provides a unified solution to the many varied 
communications and messaging devices used daily by mobile professionals and active 
consumers, with simple voice commands, subscribers can easily access and respond to 
all of their communications and messaging media in the same session. 
5 The system takes inbound calls and contacts the subscriber, using call transfer 

or follow me features, and advises the subscriber of the call, the number of the calling 
party and/or the callers identity from the subscribers contact list. If the subscriber is on 
the telephone, the system will whisper the pending call information, giving the 
subscriber the option of taking the call or sending the caller into voice mail. The 

1 0 subscriber can also set priorities for certain calls which the system will follow, 

permitting selected calls to be put on call waiting, transferred or directed through call 
forwarding, while other calls are direct to voice mail messaging. Subscribers can 
conveniently make calls from their contact list by voice commands giving the, contacts 
name and, if applicable, the location to be called (i.e. "home," "office," "cellular," etc.) 

15 Subscribers can access and play back their voice mail from any telephone or 

from their personal web page. 

Voice mail messages can be saved, retrieved, deleted or rerouted to other 
individuals, groups or broadcast and voice responses can be returned immediately to the 
caller. Also, e-mail messages can be viewed on the subscriber's personal home page or 

20 the system will read the e-mail to the subscriber from any telephone using text-to- 
speech technology. Subscribers can immediately respond to e-mail with voice 
messages, marking, saving or deleting messages during the same session. E-mail 
messages can also be sent to any fax machine. Further, subscribers can immediately 
respond to faxes rather than waiting to retrieve copies and delay responses. Incoming 

25 faxes are received, the subscriber is notified of the arrival of the new fax, the fax can 
then be stored for later viewing and/or redirected to any fax machine or email address 
from the subscribers contact list or any other number. Subscribers can also be notified 
of any incoming communications and messages by pager, whether an inbound call, 
voice mail, e-mail or fax. 

30 The system will automate conference calling and eliminate the need for 

conference call operators or complex, confusing PBX systems. The subscriber remains 
in control and can add or drop callers, mute the call and otherwise control all aspects of 
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the conference call. By pre-scheduling conference calls by date and time, subscribers 
can notify participants, giving them an 800 number to call and join the conference. Up 
to 32 participants can be included in the present conference call feature. 

The system further acts as an efficient secretary retaining complete contact logs 
of all call statistics. The subscriber maintains the contact list using simple voice or 
keyboard commands and can establish contact groups for broadcast communications 
and setting up conference calls. The contact database is easily accessible to all features. 

The system includes a feature which enables subscribers to retrieve on demand 
or at predetermined intervals selected information from the internet or online service 
providers, allowing subscribers to establish "filter and forward" criteria specifying the 
type of information desired. A search engine will then retrieve the requested 
information, transmit the information to the system platform and notify the subscriber 
by page, telephone or other desired means. The system will provide direct access to 
news, weather, sports, financial, travel and other custom content directly from a 
computer or any telephone. The subscriber will then access the information by all 
available options, including text-to-speech capabilities. 

The system provides long distance and international calling over the internet 
through the subscriber's personal home page, thereby significantly reducing the costs of 
long distance and international calling. The system further provides video conferencing 
features. 

It is to be understood that the form of this invention is merely a preferred 
embodiment, various changes may be made in the function and arrangement of parts; 
equivalent means may be substituted for those illustrated and described; and certain 
features may be used independently from others without departing from the spirit and 
scope of the invention as defined in the following claims. 
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WHAT IS CLAIMED IS: 

1 . A computer and telecommunications network for receiving, sending and 
managing information from a subscriber to the network and from the network to a 
subscriber comprising: 

5 at least one cluster, said cluster containing at least one voice server, said voice 

server containing telephony, speech recognition, text-to-speech and conferencing 
functions, such that said subscriber can access said cluster by a standard telephone 
connection or by a internet connection; 

at least one database server, said database server being connected to said cluster 
10 and containing contact lists and administrative data, such that said subscriber can 
manipulate and manage said data; 

at least one file server, said file server being connected to said cluster; and 
a web server, said web server being connected to said cluster such that said 
subscriber can access said network by connecting to said web server via the internet; 
15 wherein said network can receive a message from said telephone connection or 

said internet connection and transmit said message to said subscriber by said telephone 
or internet connection, and said network can receive a message from said subscriber by 
telephone connection or internet connection and transmit said message by telephone 
connection or said internet connection based on commands received from said 
20 subscriber. 

2. The computer and telecommunications network described in claim 1, wherein 
the information received or transmitted by said subscriber comprises telephone calls, e- 
mail, faxes, conference calls and voice-mail. 

3. The computer and telecommunications network described in claim 1, wherein 
25 manipulating and managing said data comprises creating and modifying a contact list 

4. The computer and telecommunications network described in claim 3, wherein 
manipulating and managing said data comprises utilizing said contact list to place 
telephone calls, send e-mail, voice-mail and faxes, and create or modify conference 
calls. 

30 5. A method for receiving, sending and managing information between a 

computer and telecommunications network and a subscriber comprising the steps of: 
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providing at least one cluster, said cluster containing at least one voice server, 
said voice server containing telephony, speech recognition, text-to-speech and 
conferencing functions, such that said subscriber can access said cluster by a standard 
telephone connection or by a internet connection; 

providing at least one database server, said database server being connected to 
said cluster and containing contact lists and administrative data, such that said 
subscriber can manipulate and manage said data; 

providing at least one file server, said file server being connected to said cluster; 

and 

providing a web server, said web server being connected to said cluster such 
that said subscriber can access said network by connecting to said web server via the 
internet; 

receiving a message from said telephone connection or said internet connection 
transmitting said message to said subscriber by said telephone or internet connection, 
based on commands received by the network from said subscriber. 

6. The computer and telecommunications network described in claim 5, wherein 
the information received or transmitted by said subscriber comprises telephone calls, e- 
mail, faxes, conference calls and voice-mail. 

7. The computer and telecommunications network described in claim 5, wherein 
manipulating and managing said data comprises creating and modifying a contact list. 

8. The computer and telecommunications network described in claim 7, wherein 
manipulating and managing said data comprises utilizing said contact list to place 
telephone calls, send e-mail, voice-mail and faxes, and create or modify conference 
calls. 

9. A method for receiving, sending and managing information between a 
computer and telecommunications network and a subscriber comprising the steps of: 

providing at least one cluster, said cluster containing at least one voice server, 
said voice server containing telephony, speech recognition, text-to-speech and 
conferencing functions, such that said subscriber can access said cluster by a standard 
telephone connection or by a internet connection; 
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providing at least one database server, said database server being connected to 
said cluster and containing contact lists and administrative data, such that said 
subscriber can manipulate and manage said data; 

providing at least one file server, said file server being connected to said cluster; 

and 

providing a web server, said web server being connected to said cluster such 
that said subscriber can access said network by connecting to said web server via the 
internet; 

receiving a message from said subscriber by telephone connection or said 
internet connection; and 

transmitting said message by said telephone or internet connection, based on 
commands received by the network from said subscriber. 

10. The computer and telecommunications network described in claim 9, 
wherein the information received or transmitted by said subscriber comprises telephone 
calls, e-mail, faxes, conference calls and voice-mail. 

1 1 . The computer and telecommunications network described in claim 9, 
wherein manipulating and managing said data comprises creating and modifying a 
contact list. 

12. The computer and telecommunications network described in claim 11, 
wherein manipulating and managing said data comprises utilizing said contact list to 
place telephone calls, send e-mail, voice-mail and faxes, and create or modify 
conference calls. 
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